Directional focus navigation

ABSTRACT

A rendering engine that supports directional focus navigation, in response to a directional input, shifts the focus to an appropriate focusable element. The rendering engine determines one or more focusable elements that are candidates to receive the focus based on the received directional input and based on the relative display locations of the focusable elements. The rendering engine selects the most appropriate candidate focusable element based on each candidate element&#39;s display location in relation to the display location of the element that currently has focus.

RELATED APPLICATIONS

This application claims priority under 35 U.S.C. §120 as a continuationof U.S. patent application Ser. No. 10/293,447, filed Nov. 13, 2002, andtitled “Directional Focus Navigation”, the entire disclosure of which ishereby incorporated by reference in its entirety.

TECHNICAL FIELD

This invention relates to focus navigation and, in particular, todirectional focus navigation.

BACKGROUND

A graphical user interface (GUI) may include multiple elements that canreceive input. The target of the input is said to have focus, andelements eligible for focus within a document are called focusableelements. In a GUI, input that is received is processed according to anelement that currently has focus. For example, if a GUI contains twoeditable text boxes, when a user enters text through a keyboard, thetext is rendered in the text box that currently has focus.

In a GUI that receives input from a pointing device, such as a mouse,the currently focused element is determined based on the location of acursor or pointer when a user submits a select command (e.g., clicks amouse button). If the cursor or pointer is over a text box when the userclicks the mouse button, then that text box is given focus and futureinput, such as through a keyboard, is directed to the text box until thefocus is moved to another element.

In a GUI that receives input from devices without pointing mechanisms,such as keyboards or remote controls, an alternative mechanism isimplemented to allow a user to specify an element to be given focus, andthus, the target of keyboard (or other device) input.

Browser or other rendering engine applications that provide a mechanismfor displaying documents that include focusable elements (e.g.,hypertext markup language (HTML) documents, dynamic HTML documents, orweb pages) are implemented to support navigation between focusableelements in a document by way of a mouse or other similar pointingdevice. Some such documents also support navigation through thefocusable elements of the document according to a tab order that definesa sequential order that determines which element is focused next when auser presses a tab key.

Recently introduced technology allows web pages (and potentially othertypes of documents that may include focusable elements) to be renderedusing devices other than computers. These other devices may include, butare not limited to, a television set, a cellular phone display, and amulti-modal device, such as the NOKIA 9110 COMMUNICATOR. Such devices donot typically include a pointer input device, such as a mouse.

Rendering engines implemented on such devices may be programmed toreceive an input that behaves like the press of a tab key to allow auser to move sequentially from one focusable element in a document toanother. However, cascading style sheets, dynamic HTML, and the likesupport dynamic rendering of documents in such a way that the visuallayout of a rendered document may differ from the layout that wasintended when the document was generated and possibly, when the taborder of focusable elements in the document was established.Accordingly, moving from one focusable element to another eithersequentially or based on an established tab order may not result invisually logical focus navigation, and it is desirable for a renderingengine to support directional focus navigation based on directionalinputs, such as an arrow key press.

SUMMARY

Directional focus navigation is described. A browser or other renderingengine that supports directional focus navigation determines focusableelements that are candidates to receive focus based on a directionalinput and a display layout of the focusable elements. After candidatefocusable elements are identified, the rendering engine determines themost appropriate candidate, and moves the focus to that element.

BRIEF DESCRIPTION OF THE DRAWINGS

The same numbers are used throughout the drawings to reference likefeatures and components.

FIG. 1 a illustrates a portion of an exemplary method for implementingdirectional focus navigation.

FIG. 1 b illustrates a portion of an exemplary method for implementingdirectional focus navigation.

FIG. 2 illustrates a view of a document with three focusable elements.

FIG. 3 illustrates a view of a document with several focusable elementsand a table structure.

FIG. 4 illustrates a view of a document with several focusable elements.

FIG. 5 illustrates a view of a document that includes focusable areas.

FIG. 6 a illustrates a view of a scrollable document with severalfocusable elements.

FIG. 6 b illustrates a view of a scrollable document with severalfocusable elements.

FIG. 6 c illustrates a view of a scrollable document with severalfocusable elements.

FIG. 7 a illustrates a view of a parent document that includes ascrollable element.

FIG. 7 b illustrates a view of a parent document that includes ascrollable element.

FIG. 8 illustrates a view of two documents with focusable elementsdisplayed using frames.

FIG. 9 illustrates components of an exemplary environment in whichdirectional focus navigation may be implemented.

FIG. 10 illustrates select components of a client device implemented tosupport directional focus navigation.

DETAILED DESCRIPTION

The following discussion is directed to directional focus navigationamong IS focusable elements of a document. Directional focus navigationas described herein may be implemented using a browser or otherinterface or rendering engine that renders documents (e.g., HTML ordynamic HTML) that may contain focusable elements to provide a mechanismfor navigating among those focusable elements using directional inputs(e.g., left, right, up, and down arrow keys) instead of a traditionalpointing device such as a mouse. For example, directional focusnavigation may be implemented to support web site navigation using atelevision remote control, when an HTML rendering engine is implementedas part of a television or television set-top box. In another example,directional focus navigation may be implemented to support web sitenavigation using a cellular telephone or multi-modal device.

Method for Directional Focus Navigation

Directional focus navigation may be described in the general context ofcomputer-executable instructions, such as application modules, beingexecuted by a computer. Generally, application modules include routines,programs, objects, components, data structures, etc. that performparticular tasks or implement particular abstract data types.Directional focus navigation may be implemented using any number ofprogramming techniques and may be implemented in local computingenvironments or in distributed computing environments where tasks areperformed by remote processing devices that are linked through variouscommunications networks based on any number of communication protocols.In such a distributed computing environment, application modules may belocated in both local and remote computer storage media including memorystorage devices.

FIGS. 1 a and 1 b illustrate a method 100 for implementing directionalfocus navigation among focusable elements and/or focusable areas of arendered document. FIG. 1 a illustrates a portion of the method 100 thatpertains to directional focus navigation when candidate focusableelements are within a display area associated with a document. FIG. 1 billustrates a portion of the method 100 that pertains to directionalfocus navigation when candidate focusable elements are not within adisplay area associated with a document.

Documents that may be rendered as described herein can be represented bya hierarchical tree structure that illustrates the relationships betweenelements of the document. For example, the document itself may berepresented by an element (e.g., an HTML document element) that containsthe other elements of the document, such as buttons, hyperlinks, tables,and so on. Per the hierarchical structure, elements within the documentelement may also contain other elements. For example, a table elementmay contain cell elements. When a document is first rendered, the focusis given to some element associated with the document. That element maybe the document element, or it may be any other focusable elementcontained within the document element

At block 102, a browser (or other type of rendering application)receives an indication of a directional input. For example, theindication is received when a user presses an arrow key (e.g., up, down,left, or right) on a keyboard, remote control, phone keypad, or othersuch device.

At block 104 the browser determines whether or not the current focus ison an element associated with the currently rendered document. If thebrowser determines that the current focus is not on an elementassociated with the document (the “No” branch from block 104), then atblock 106 the browser identifies the first focusable element. The methodcontinues at block 108, setting the focus to the identified firstfocusable element. In the described implementation, the first focusableelement of a document is a focusable element that is a child of theelement representing the entire document that is displayed above and tothe left of any other focusable elements that are children of theelement representing the entire document.

If the browser determines that there is a currently focused elementassociated with the document (the “Yes” branch from block 104), then atblock 110, the browser identifies the element that currently has thefocus.

At block 112 the browser determines whether a focus candidate exists. Inthe described implementation, a focus candidate is a focusable elementthat is within the viewable area of the document and displayed at alocation that is indicated by the received directional input in relationto the display location of the currently focused element. If it isdetermined that there is not a focus candidate associated with thecurrent document (the “No” branch from block 112), then the methodcontinues at block 122 (shown in FIG. 1 b). On the other hand, if it isdetermined that there is a focus candidate associated with the currentdocument (the “Yes” branch from block 112), then the method continues atblock 114.

At block 114 the browser selects a focus candidate that is associatedwith the current document.

At block 116 the browser determines whether or not the selected elementis the most appropriate element to receive the focus based on thedisplay location of the selected element in relation to the displaylocation of the element that currently has the focus. If the selectedelement is an appropriate element to receive the focus based on itsdisplay location, and no other focusable element has been identified asan appropriate element, then the selected element is identified as themost appropriate element thus far. If the selected element is anappropriate element to receive the focus based on its display location,and another focusable element has previously been identified as the mostappropriate element thus far, then the browser compares the locations ofthe two identified appropriate elements to determine which one is themost appropriate element thus far. If the browser determines that theselected element is the most appropriate element to receive the focus(the “Yes” branch from block 116), then the method continues at block118. On the other hand, if the browser determines that the selectedelement is not the most appropriate element to receive the focus (the“No” branch from block 116), then the method continues at block 120.

At block 118, the browser identifies the selected element as the elementto which the focus will be given.

At block 120 the browser determines whether or not there is anotherfocusable element within the viewable area of the document that has notbeen selected as described with reference to block 114. If the browserdetermines that there is another focusable element that has not beenselected (the “Yes” branch from block 120), then the method continues atblock 114, as described above. On the other hand, if the browserdetermines that there are no other focusable elements that have not beenselected (the “No” branch from block 120), then the method continues atblock 108, setting the current focus to the focusable element identifiedas described above with reference to block 118.

As described above with reference to block 112, if the browserdetermines that there is not another focusable element within theviewable area of the document, then at block 122 (shown in FIG. 1 b),the browser determines whether or not the current document is scrollablein the direction indicated by the received directional input (describedabove with reference to block 102). If the document is scrollable in thedirection indicated (the “Yes” branch from block 122), then at block 124the browser scrolls the document in the direction indicated. On theother hand, if the document is not scrollable in the direction indicated(the “No” branch from block 122), then the method continues at block126.

At block 126 the browser determines whether the document associated withcurrently focused element is a child document associated with anotherparent document. If the browser determines that there is not a parentdocument (the “No” branch from block 126), then at block 128, the methodterminates with the focus remaining on the currently focused element. Onthe other hand, if the browser determines that there is a parentdocument (the “Yes” branch from block 126), then the method continues atblock 130.

At block 130 the browser determines whether there are focusable elementswithin the viewable area of the parent document that are appropriatebased on the display location of the elements in relation to the displaylocation of the element with the current focus based on the directionindicated by the directional input. If there are no appropriatefocusable elements within the viewable area of the parent (the “No”branch from block 130), then at block 132 the browser sets the focus tothe parent document. On the other hand, if there is at least oneappropriate focusable element within the viewable area of the parentdocument (the “Yes” branch from block 130), the method continues atblock 134.

At block 134 the browser identifies the most appropriate focusableelement in the parent document to receive the focus.

At block 136 the browser sets the current focus to the elementidentified as described with reference to block 134.

Horizontal Focus Navigation

Horizontal focus navigation pertains to the moving of focus from oneelement to another element in response to a left or right directionalinput. In an implementation of directional focus navigation, when movingfocus horizontally, preference is given to a focusable element with adisplay position that has more vertical overlap with the displayposition of the currently focused element over a focusable element thathas less vertical overlap with the display position of the currentlyfocused element. Details of a horizontal focus navigation implementationwill be described below with reference to FIG. 2.

Right Directional Input

FIG. 2 illustrates a view of a document that includes three focusableelements. Box 202 represents the viewable area supported by a renderingengine. Box 204 represents the document. Hyperlink 206 is the elementthat currently has the focus (indicated by the dashed box). When therendering engine receives a s right directional input (e.g., a userpresses a right arrow key), the rendering engine determines which, ifany, of the other focusable elements (i.e., hyperlink 208 or hyperlink210) is the most appropriate element to receive the focus next. (Thisprocess is described above with reference to blocks 102-120 shown inFIG. 1 a.) The following mathematical definitions will be used below inhorizontal focus navigation equations that may be used in animplementation of a rendering engine that supports directional focusnavigation:

-   -   F=element that has the current focus    -   E=focusable element that is being evaluated    -   W=focusable element that is currently identified as the most        appropriate focusable element to receive the focus so far    -   x.left=the left coordinate of the display location of element x    -   x.right=the right coordinate of the display location of element        x    -   x.top=the top coordinate of the display location of element x    -   x.bottom=the bottom coordinate of the display location of        element x    -   x.width=x.right−x.left    -   x.height=x.bottom−x.top    -   x.mid=x.left+(x.width)/2    -   x.vertoverlap=min(z, x.height, F.height), where:        -   z=min(x.bottom−F.top, F.bottom−x.top)    -   x.% vertoverlap=(x.vertoverlap)/(x.height)

The above (and all following) equations are written using the assumptionthat the display area can be mapped to a two-dimensional grid in whichthe upper-left-most corner has (x,y) coordinates (0,0).

If a focusable element E meets the following criteria, then E isconsidered a focus candidate (an appropriate element to receive focus)when the directional input indicates moving focus to the right:

-   -   E is within the displayable area of the document.    -   E.mid>F.mid        -   (The midpoint of E is to the right of the midpoint of F.)    -   (F.bottom−E.bottom)<=LaxHeight1 OR        -   (F.top−E.top)<=LaxHeight1, where LaxHeight1 is a            configurable tolerance value used to compare the vertical            display locations of element E and element F. In the            described implementation, LaxHeight1 is derived from            F.height and E.height, such as LaxHeight1=(max(F.height,            E.height))/4.        -   (E is at the same level or below F, or E is above F by no            more than LaxHeight1.)

Comparing the horizontal midpoints (E.mid>F.mid) provides a mechanismfor handling the special case in which two focusable elements haveoverlapping display areas.

Given the document illustrated in FIG. 2, hyperlinks 208 and 210 bothqualify as appropriate elements to receive focus when a rightdirectional input is received while the current focus is on hyperlink206.

As described above with reference to block 116 shown in FIG. 1, thebrowser compares the elements that qualify as appropriate elements toreceive the focus to determine the most appropriate element to receivethe focus. In this implementation, element E is determined to be themost appropriate element to receive the focus when a right directionalinput is received if any of the following is true:

-   -   E.vertoverlap>=0 AND W.vertoverlap<0        -   (E vertically overlaps F, but W does not vertically overlap            F.)    -   E.vertoverlap<0 AND W.vertoverlap<0 AND        E.vertoverlap>W.vertoverlap        -   (Neither E nor W vertically overlaps F, but E is vertically            closer to F.)    -   abs(E.vertoverlap−W.vertoverlap)<=LaxHeight2 AND E.left<W.left,        where LaxHeight2 is a configurable tolerance value used to        compare the vertical overlap of E and W with respect to F. In        the described implementation, LaxHeight2 is derived from the        heights of each of the elements involved in the comparison, such        as LaxHeight2=(min(F.height, E.height, W.height))/3.        -   (E and W have similar overlap values, but E is horizontally            closer to F.)    -   E.vertoverlap>(W.vertoverlap+LaxHeight2) AND ((E.left<W.left)        -   (E has a greater vertical overlap and E is horizontally            closer to F.)    -   E.vertoverlap>(W.vertoverlap+LaxHeight2) AND (E. %        vertoverlap>W. % vertoverlap)        -   (E has a greater vertical overlap and E has a greater            percentage vertical overlap.)    -   (E.left<W.left) AND (E. % vertoverlap>W. % vertoverlap)        -   (E is horizontally closer to F and E has a greater            percentage vertical overlap.)

In this implementation, preference is given to an element thatvertically overlaps the currently focused element because most documentshave a visual layout that suggests reading the document first from leftto right and then from top to bottom. In alternate implementations, thespecific algorithm can be modified to support other document layouts.Given the calculations above, in the document shown in FIG. 2, hyperlink208 is determined to be the most appropriate element to receive thefocus in response to a right directional input when hyperlink 206 is theelement that currently has the focus because although hyperlink 210 ishorizontally closer to the currently focused element, hyperlink 208vertically overlaps hyperlink 206 and hyperlink 210 does not verticallyoverlap hyperlink 206.

Left Directional Input

Similarly, if a focusable element E meets the following criteria, then Eis considered an appropriate element to receive focus when thedirectional input indicates moving focus to the left:

-   -   E is within the displayable area of the document.    -   E.mid<F.mid        -   (The midpoint of E is to the left of the midpoint of F.)    -   (E.bottom−F.bottom)<=LaxHeight1 OR (E.top−F.top)<=LaxHeight1,        where LaxHeight1 is a configurable tolerance value used to        compare the vertical display locations of element E and        element F. In the described implementation, LaxHeight1 is        derived from F.height and E.height, such as        LaxHeight1=(max(F.height, E.height))/4.        -   (E is at the same level or above F, or E is below F by no            more than LaxHeight1.)

Element E is determined to be the most appropriate element to receivethe focus when compared to element W when a left directional input isreceived if any of the following is true:

-   -   E.vertoverlap>=0 AND W.vertoverlap<0        -   (E vertically overlaps F but W does not vertically overlap            F.)    -   E.vertoverlap<0 AND W.vertoverlap<0 AND        E.vertoverlap>W.vertoverlap        -   (Neither E nor W vertically overlap F, but E is vertically            closer to F)    -   abs(E.vertoverlap−W.vertoverlap)<=LaxHeight2 AND        E.right>W.right, where LaxHeight2 is a configurable tolerance        value used to compare the vertical overlap of E and W with        respect to F. In the described implementation, LaxHeight2 is        derived from the heights of each of the elements involved in        this comparison, such as LaxHeight2=(min(F.height, E.height,        W.height))/3.        -   (E and W have similar overlap values, but E is horizontally            closer to F)    -   E.vertoverlap>(W.vertoverlap+LaxHeight2) AND ((E.right>Wright)        -   (E has a greater vertical overlap and E is horizontally            closer to F.)    -   E.vertoverlap>(W.vertoverlap+LaxHeight2) AND (E. %        vertoverlap>W. % vertoverlap)        -   (E has a greater vertical overlap and E has a greater            percentage vertical overlap.)    -   (E.right>W.right) AND (E. % vertoverlap>W. % vertoverlap)        -   (E is horizontally closer to F and E has a greater            percentage vertical overlap.)

Vertical Focus Navigation

Vertical focus navigation pertains to the moving of focus from oneelement to another element in response to an up or down directionalinput. In an implementation of directional focus navigation, when movingfocus vertically, preference is first given to a focusable element thatis structurally close to the element that currently has the focus.Preference is next given to a focusable element that has horizontalcoordinates closest to the horizontal coordinates of the last elementthat got focus by means other than an up or down directional input.Details of a vertical focus navigation implementation will be describedbelow with reference to FIGS. 3 and 4.

Down Directional Input

FIG. 3 illustrates a view of a document that includes several focusableelements, including a table structure. Box 302 represents the viewablearea supported by the browser. Box 304 represents the document. Table306 is table structure within document 304. Cell4 308 is the elementthat currently has the focus (indicated by the dashed box) and cell7 310is another cell in table 306 that is also a focusable element. Button312 is another focusable element within document 304 that is not withintable structure 306.

Documents that can be rendered using a rendering engine can be definedin terms of hierarchical tree structure of elements. For example, thedocument 304 illustrated in FIG. 3, can be described in terms of a treestructure in which the document itself is the root element of the treestructure. Table 306 and button 312 are child elements of the rootelement. Cell4 308 and cell7 310 are focusable elements that arechildren of the table element in the tree structure.

When the rendering engine receives a down directional input (e.g., auser presses a down arrow key), the rendering engine determines which,if any, of the other focusable elements (i.e., hyperlink 310 or button312) is the most appropriate element to receive the focus next. (Thisprocess is described above with reference to blocks 102-120 shown inFIG. 1 a.)

The following mathematical definitions will be used below in verticalfocus navigation equations that may be used in an implementation of arendering engine that supports directional focus navigation:

-   -   F=element that has the current focus;    -   E=focusable element that is being evaluated;    -   W=focusable element that is currently identified as the most        appropriate focusable element to receive the focus so far;    -   x.left=the left coordinate of the display location of element x;    -   x.right=the right coordinate of the display location of element        x;    -   x.top=the top coordinate of the display location of element x;    -   x.bottom=the bottom coordinate of the display location of        element x;    -   x.width=x.right−x.left;    -   x.height=x.bottom−x.top;    -   x.mid=x.top+(x.height)/2;    -   x.DistanceToCommonAncestor=the number of hierarchical levels,        according to an element tree structure, between element x and        the closest ancestor element that is common to both element x        and element F;    -   CurrentLeft=the left coordinate of the last element that got        focus by means other than an up or down directional input;    -   CurrentRight=the right coordinate of the last element that got        focus by means other than an up or down directional input; and    -   x.horizoverlap=min(x.right−CurrentLeft, CurrentRight−x.left).

The above (and all following) equations are written using the assumptionthat the display area can be mapped to a two-dimensional grid in whichthe upper-left-most corner has (x,y) coordinates (0,0).

If a focusable element E meets the following criteria, then E isconsidered an appropriate element to receive focus when the directionalinput indicates moving focus down:

-   -   E is within the displayable area of the document;    -   (F.bottom−E.top)<LaxHeight3, where LaxHeight3 is a configurable        tolerance value used to compare the vertical display locations        of element E and element F. In the described implementation,        LaxHeight3 is derived from F.height and E.height, such as        LaxHeight3=(min(F.height, E.height))/4.        -   (E is lower than F or the top edge of E is above the bottom            edge of F by no more than LaxHeight3.)

Given the document illustrated in FIG. 3, hyperlink 310 and button 312both qualify as appropriate elements to receive focus when a downdirectional input is received while the focus is on hyperlink 308.

As described above with reference to block 116 shown in FIG. 1, thebrowser compares the elements that qualify as appropriate elements toreceive the focus to determine the most appropriate element to receivethe focus. In this implementation, element E is determined to be themost appropriate element to receive the focus when a down directionalinput is received if any of the following is true:

-   -   E.DistanceToCommonAncestor>W.DistanceToCommonAncestor        -   (E is structurally closer to F.)    -   (E.DistanceToCommonAncestor=W.DistanceToCommonAncestor) AND        (W.bottom−E.top>=compareHeight) AND        (E.horizoverlap>W.horizoverlap), where compareHeight is a        configurable tolerance value used to compare the vertical        display locations of element E and element W. In the described        implementation, compareHeight is derived from the heights of        element E and element W, such as compareHeight=(min(E.height,        W.height))/4.        -   (E is structurally equal to W, E is vertically closer to F            (or the vertical distance between E and F is approximately            equal to the vertical distance between W and F), and E has            more horizontal overlap with the element that had focus when            up/down navigation began than does W.)    -   (E.DistanceToCommonAncestor=W.DistanceToCommonAncestor) AND        (W.bottom−E.top>=compareHeight AND (W.width>=E.width) AND        (W.left<=E.left), where compareHeight is a configurable        tolerance value used to compare the vertical display locations        of element E and element W. In the described implementation,        compareHeight is derived from the heights of element E and        element W, such as compareHeight=(min(E.height, W.height))/4.        -   (E is structurally equal to W, E is vertically closer to F            (or the vertical distance between E and F is approximately            equal to the vertical distance between W and F), and W            horizontally contains E.)    -   (E.DistanceToCommonAncestor=W.DistanceToCommonAncestor) AND        (W.bottom−E.top>=compareHeight AND (E.width>=W.width) AND        (E.left<=W.left), where compareHeight is a configurable        tolerance value used to compare the vertical display locations        of element E and element W. In the described implementation,        compareHeight is derived from the heights of element E and        element W, such as compareHeight=(min(E.height, W.height))/4.        -   (E is structurally equal to W, E is vertically closer to F            (or the vertical distance between E and F is approximately            equal to the vertical distance between W and F), and E            horizontally contains W.)    -   (E.DistanceToCommonAncestor=W.DistanceToCommonAncestor) AND        (E.bottom<=W.top).        -   (E is structurally equal to W, and E is completely above W.)

In this implementation, preference is given first to an element that isstructurally closer to the currently focused element, as defined by thetree structure that represents the document. If a decision betweenmultiple focusable elements remains, preference is then given to anelement that is horizontally closer to the element that had focus whenup/down navigation began. Therefore, in the document shown in FIG. 3,hyperlinked Cell7 310 is determined to be the most appropriate elementto receive the focus in response to a down directional input whenhyperlinked Cell4 308 is the element that currently has the focus.Hyperlinked Cell7 310 is given preference because although focusablebutton 312 is vertically closer to hyperlink 308, hyperlinked Cell7 310is structurally closer to hyperlinked Cell4 308 based on the elementtree structure that represents document 304. That is, the commonancestor between hyperlink 308 and Cell7 310 (i.e., table 306) is closerto hyperlink 308 than the common ancestor between hyperlink 308 andbutton 312 (i.e., document 304) is to hyperlink 308. As a result,hyperlink 310 is selected as the next element to receive focus becauseit has the closest ancestor element in common with the currently focusedelement, according to an element tree structure that represents thedocument.

FIG. 4 illustrates a view of a document 402 that includes severalfocusable elements that are structurally equivalent based on an elementtree structure representation of the elements in the document. For thisdiscussion, it is assumed that hyperlink 406 is the element thatcurrently has focus (indicated by the dashed line). Furthermore, it isassumed that hyperlink 406 received the focus based on something otherthan an up or down directional input. For example, hyperlink 406 mayhave been given the focus when the document was initially rendered or inresponse to a left or right directional input. When the rendering enginereceives a down directional input (e.g., a user presses a down arrowkey), the rendering engine determines, based on the equations describedabove, the most appropriate focusable element to receive the focus inresponse to the down directional input.

Link02-Link22 are all identified as appropriate elements to receivefocus in response to the down directional input because each is withinthe focusable area of the document (indicated by box 404), the midpointof each is below the midpoint of the currently focused element, and thetop of each is below the bottom of the currently focused element.

When the focus is on element 406 and the rendering engine receives adown directional input, element 408 is given focus because it isvertically closer to the currently focused element 406 than any of theother elements that were determined appropriate for receiving focus.

When the focus is on element 408 and the rendering engine receivesanother down directional input, Link03-Link22 are all identified asappropriate elements to receive focus because they are below thecurrently focused element, 408. Because they are vertically closer tothe currently focused element, Link03 and Link 04 are given preference.To decide between Link03 and Link04, the rendering engine compares thehorizontal location of each with the horizontal location of the elementthat had focus when the up/down navigation began, which is hyperlink406. Because Link04 is horizontally closer to hyperlink 406 than isLink03, hyperlink 410 (Link04) is given the focus in response to a downdirectional input received when the focus is on hyperlink 408.

Similarly, when the focus is on element 410 and the rendering enginereceives another down directional input, Link05-Link22 are allidentified as appropriate elements to receive focus because they arebelow the currently focused element, 410. Because they are verticallycloser to the currently focused element, Link05-Link 07 are givenpreference. To decide between Link05, Link06, and Link07, the renderingengine compares the horizontal location of each with the horizontallocation of the element that had focus when the up/down navigationbegan, which is hyperlink 406. Because Link07 is horizontally closer tohyperlink 406 than are Link05 or Link06, hyperlink 412 (Link07) is giventhe focus in response to a down directional input received when thefocus is on hyperlink 410.

By the same rationale, if the rendering engine receives another downdirectional input while focus is on element 412, focus will move toelement 414. When focus is on element 414 and the rendering enginereceives another down directional input, focus moves to element 416.Finally, when focus is on element 414 and the rendering engine receivesanother down directional input, focus move to element 418.

Up Directional Input

Similarly, if a focusable element E meets the following criteria, then Eis considered an appropriate element to receive focus when thedirectional input indicates moving focus up:

-   -   E is within the displayable area of the document;    -   (E.bottom−F.top)<LaxHeight3, where LaxHeight3 is a configurable        tolerance value used to compare the vertical display locations        of element E and element F. In the described implementation,        LaxHeight3 is derived from the heights of element E and element        F, such as LaxHeight3=(min(F.height, E.height))/4.        -   (E is above F or the lower edge of E is below the top edge            of F by no more than LaxHeight3.)

Element E is determined to be the most appropriate element to receivethe focus when an up directional input is received if any of thefollowing is true:

-   -   E.DistanceToCommonAncestor>W. DistanceToCommonAncestor (E is        structurally closer to F.)    -   (E.DistanceToCommonAncestor=W.DistanceToCommonAncestor) AND        (E.bottom−W.top>=compareHeight) AND        -   (E.horizoverlap>W.horizoverlap), where compareHeight is a            configurable tolerance value used to compare the vertical            display locations of element E and element W. In the            described implementation, compareHeight is derived from the            heights of element E and element W, such as            compareHeight=(min(E.height, W.height))/4.        -   (E is structurally equal to W, E is vertically closer to F            (or the vertical distance between E and F is approximately            equal to the vertical distance between W and F), and E has            more horizontal overlap with the element that had focus when            up/down navigation began than does W.)    -   (E.DistanceToCommonAncestor=W.DistanceToCommonAncestor) AND        (E.bottom−W.top>=compareHeight) AND (W.width>=E.width) AND        (W.left<=E.left), where compareHeight is a configurable        tolerance value used to compare the vertical display locations        of element E and element W. In the described implementation,        compareHeight is derived from the heights of element E and        element W, such as compareHeight=(min(E.height, W.height))/4.        -   (E is structurally equal to W, E is vertically closer to F            (or the vertical distance between E and F is approximately            equal to the vertical distance between W and F), and W            horizontally contains E.)    -   (E.DistanceToCommonAncestor=W.DistanceToCommonAncestor) AND        (E.bottom−W.top>=compareHeight) AND (E.width>=W.width) AND        (E.left<=W.left), where compareHeight is a configurable        tolerance value used to compare the vertical display locations        of element E and element W. In the described implementation,        compareHeight is derived from the heights of element E and        element W, such as compareHeight=(min(E.height, W.height))/4.        -   (E is structurally equal to W, E is vertically closer to F            (or the vertical distance between E and F is approximately            equal to the vertical distance between W and F), and E            horizontally contains W.)    -   (E.DistanceToCommonAncestor=W.DistanceToCommonAncestor) AND        (E.top>=W.bottom).        -   (E is structurally equal to W, and E is completely below W.)

Directional Focus Navigation Among Focusable Areas

A rendering engine as described herein can be implemented to supportdirectional focus navigation across arbitrarily shaped areas, such asareas defined for an image map. FIG. 5 illustrates a rendered view of adocument that includes an image map of the United States of America inwhich each state is a focusable area. Area 502 is the area thatcurrently has the focus (indicated by the solid outline). When therendering engine receives a directional input (e.g., a user presses aright arrow key), the rendering engine determines which, if any, of theother focusable areas (i.e., the other focusable areas in the image map)is the most appropriate area to receive the focus next. (This process isdescribed above with reference to blocks 102-120 shown in FIG. 1 a.)

The following mathematical definitions will be used below in focusnavigation equations that may be used in an implementation of arendering engine that supports directional focus navigation across areasof arbitrary shape:

-   -   F=area that has the current focus    -   E=focusable area that is being evaluated    -   W=focusable area that is currently identified as the most        appropriate focusable area to receive the focus so far    -   x.left=the left coordinate of the left-most point of area x    -   x.right=the right coordinate of the right-most point of area x    -   x.top=the top coordinate of the top-most point of area x    -   x.bottom=the bottom coordinate of bottom-most point of area x    -   x.width=x.right−x.left    -   x.height=x.bottom−x.top

The above (and all following) equations are written using the assumptionthat the display area can be mapped to a two-dimensional grid in whichthe upper-left-most corner has (x,y) coordinates (0,0).

Right Directional Input

If a focusable area E meets the following criteria, then E is consideredan appropriate area to receive focus when the directional inputindicates moving focus to the right:

-   -   E is within the displayable area of the document.    -   E.right−F.right>=LaxWidth, where LaxWidth is a configurable        tolerance value used to compare the horizontal display locations        of element E and element F. In the described implementation,        LaxWidth is derived from the widths of element E and element F,        such as LaxWidth=(min(E.width, F.width))/5.        -   (The right-most point of E is to the right of the right-most            point of F by a distance of at least LaxWidth.)

As described above with reference to block 116 shown in FIG. 1, thebrowser compares the elements (or areas) that qualify as appropriateelements (or areas) to receive the focus to determine the mostappropriate element (or area) to receive the focus. In thisimplementation, area E is determined to be the most appropriate area toreceive the focus when a right directional input is received if thedistance between a representative point of E and a representative pointof F is shorter than the distance between the representative point of Fand any of the other appropriate elements or areas to receive focus.

In the described implementation, the representative point of an area hasa horizontal coordinate and a vertical coordinate. In oneimplementation, the horizontal coordinate of the representative point ofan area is the average of the horizontal coordinates of all of thepoints of the area, while the vertical coordinate of the representativepoint is the average of the vertical coordinates of all of the points ofthe area. In an alternate implementation, the representative point of anarea may be the midpoint of a particular side of the area. Although thedescribed implementation suggests representative points that arecalculated as average points, it is recognized that any number ofmethods and/or calculations may be used to determine a representativepoint associated with an area.

Left Directional Input

If a focusable area E meets the following criteria, then E is consideredan appropriate area to receive focus when the directional inputindicates moving focus to the left:

-   -   E is within the displayable area of the document.    -   F.left−E.left>=LaxWidth, where LaxWidth is a configurable        tolerance value used to compare the horizontal display locations        of element E and element F. In the described implementation,        LaxWidth is derived from the widths of the element E and element        F, such as LaxWidth=(min(E.width, F.width))/5.        -   (The left-most point of E is to the left of the left-most            point of F by a distance of at least LaxWidth.)

In this implementation, area E is determined to be the most appropriatearea to receive the focus when a left directional input is received ifthe distance between the representative point of E and therepresentative point of F is shorter than the distance between therepresentative point of F and any of the other appropriate elements orareas to receive focus.

Down Directional Input

If a focusable area E meets the following criteria, then E is consideredan appropriate area to receive focus when the directional inputindicates moving focus down:

-   -   E is within the displayable area of the document.    -   E.bottom−F.bottom>=LaxHeight, where LaxHeight is a configurable        tolerance value used to compare the vertical display locations        of element E and element F. In the described implementation,        LaxHeight derived from the heights of element E and element F,        such as LaxHeight=(min(E.height, F.height))/4.        -   (The bottom-most point of E is below the bottom-most point            of F by a distance of at least LaxHeight.)

As described above with reference to block 116 shown in FIG. 1, thebrowser compares the elements (or areas) that qualify as appropriateelements (or areas) to receive the focus to determine the mostappropriate element (or area) to receive the focus. Similar to theimplementation described above for horizontal focus navigation acrossarbitrarily shaped areas, area E is determined to be the mostappropriate area to receive the focus when a down directional input isreceived if the distance between the representative point of E and avertical representative point of F is shorter than the distance betweenthe vertical representative point of F and any of the other appropriateelements or areas to receive focus.

In the described implementation, the representative point of an area, asdescribed above, has a horizontal coordinate and a vertical coordinate.In the described implementation, when comparing focus candidates inresponse to vertical directional input (e.g., up or down), thehorizontal component of the representative point of F is derived fromthe horizontal coordinate of the representative point of the elementthat had the focus when the first of the current sequence of up and/ordown directional inputs was received.

By defining the horizontal coordinate of the vertical representativepoint of F in this way, priority is given to areas and/or elements thatare horizontally closest to the area or element that had focus when auser first submitted an up or down directional input.

Up Directional Input

If a focusable area E meets the following criteria, then E is consideredan appropriate area to receive focus when the directional inputindicates moving focus Up:

-   -   E is within the displayable area of the document.    -   F.top−E.top>=LaxHeight, where LaxHeight is a configurable        tolerance value used to compare the vertical display locations        of element E and element F. In the described implementation,        LaxHeight is derived from the heights of element E and element        F, such as LaxHeight=(min(E.height, F.height))/4.        -   (The top-most point of E is above the top-most point of F by            a distance of at least LaxHeight.)

Area E is determined to be the most appropriate area to receive thefocus when an up directional input is received if the distance betweenthe representative point of E and a vertical representative point of Fis shorter than the distance between the vertical representative pointof F and any of the other appropriate elements or areas to receivefocus.

Focus Navigation in Scrollable Documents

FIG. 6 a illustrates a scrollable document that includes focusableelements. Window 602 represents a display area, box 604 represents thedocument. The document 604 is vertically larger than the display area602. To view portions of the document 604 that are outside of thedisplay area 602, the document can be scrolled.

In typical browser implementations, in response to directional input(e.g., an arrow key press), if the document currently being rendered isscrollable, then the browser scrolls the document in the directionindicated by the directional input. In the described browserimplementation that supports directional focus navigation, whendirectional input is received, the browser can move focus from onefocusable element to another, as described above, or can cause thedocument to scroll in the direction indicated by the directional input.

In the example illustrated in FIG. 6 a, if focus is initially onhyperlink 606 and the rendering engine receives a down directionalinput, focus is moved from hyperlink 606 to hyperlink 608. If therendering engine receives another down directional input, focus movesfrom hyperlink 608 to hyperlink 610. If the rendering engine thenreceives another down directional input, focus does not move fromhyperlink 610 because there is not another focusable element belowhyperlink 610 and within the displayable area. Because there is notanother focusable element below and within the displayable area, thebrowser scrolls the document based on the down directional input.

FIG. 6 b illustrates the scrollable document that is described abovewith respect to FIG. 6 a, after the above-described down directionalinputs are received by the browser and the browser has scrolled thedocument until hyperlink 612 is within the displayable area 602. Iffocus is on hyperlink 610 and the rendering engine receives another downdirectional input, then focus is moved to hyperlink 612. If therendering engine then receives another down directional input, focusdoes not move from hyperlink 612 because there is not another focusableelement below hyperlink 612 and within the displayable area. Becausethere is not another focusable element below and within the displayablearea, the browser scrolls the document according to the directionalinput.

In an alternate implementation, in response to a directional input, thebrowser may both scroll and move focus. For example, if a browserreceives a down directional input, and the only focus candidate elementis located outside the viewable area, the browser may scroll thedocument in the down direction until the focus candidate is within theviewable area, and then move focus to that element.

FIG. 6 c illustrates the scrollable document that is described abovewith respect to FIGS. 6 a and 6 b, after the above-described downdirectional inputs are received by the browser and the browser hasscrolled the document until the lower-most point of the document iswithin the displayable area 602.

FIG. 7 a illustrates a document that includes focusable elements whereone of the focusable elements is scrollable and contains other focusableelements. Document 702 includes focusable elements 704 (a hyperlink),706 (a scrollable frame), and 708 (a button). Element 706 isrepresentative of any scrollable element, which, in HTML, may include,but is not limited to, frame elements, object elements, div elements,and iframe elements. Document 710 is displayed in scrollable frame 706and includes focusable elements 712, 714, 716, and 718 (all hyperlinks).Assuming in this example that the browser or rendering engine that isrendering document 702 receives several sequential down directionalinputs. Assuming that focus is initially on hyperlink 702 (as indicatedby the dashed line), when the browser receives the first downdirectional input, the browser identifies scrollable frame 706 as themost appropriate element to receive the focus. Because scrollable frame706 includes focusable elements, the focus moves to the first identifiedfocusable element in document 710, which is hyperlink 712.

When the browser receives a second down directional input, the focusmoves to hyperlink 714.

When the browser receives a third down directional input, the focusremains on hyperlink 714 and the browser scrolls document 710. Dependingon the amount that document 710 is scrolled, several down directionalinputs may be received before 716 is scrolled into the displayable area(indicated by the boundaries of scrollable frame 706). Once 716 iswithin the displayable area, when another down directional input isreceived, focus moves to hyperlink 716.

A similar process of receiving down directional inputs and scrolling isrepeated until hyperlink 718 is within the displayable area, and then inresponse to the next down directional input hyperlink 718 receives thefocus.

Once hyperlink 718 has focus, additional down directional inputs willcontinue to scroll document 710 (while the focus remains on hyperlink718) until the lowermost portion of document 710 is within thedisplayable area.

FIG. 7 b illustrates the visual state of document described withreference to FIG. 7 a after the described down directional inputs arereceived and processed.

If the browser receives an additional down directional input, focusmoves from hyperlink 718 to button 708. The browser determines thatthere is not another focusable element below hyperlink 718 and withinthe displayable area of document 710, and the browser determines thatdocument 710 cannot be scrolled in the down direction any furtherAccordingly, the browser identifies button 708 as the next focusableelement below scrollable frame 706 in document 710.

Focus Navigation Between Frames

FIG. 8 illustrates a rendered view of two documents that includefocusable elements that are displayed using frames. Documents 802 and804 are displayed in frames 806 and 808, respectively. Document 802includes hyperlinks 810, 812, 814, and 816. Document 804 includeshyperlink 818.

If focus is on hyperlink 812 (as indicated by the dashed line) when abrowser application that is rendering the document receives a downdirectional input, focus moves to hyperlink 814.

However, if focus is on hyperlink 812 when a browser application that isrendering the document receives a right directional input, focus movesto hyperlink 818 because even though it is associated with a differentdocument, it is the first focusable element that has a display locationthat is to the right of hyperlink 812.

Allowing Standard Behavior in Focusable Elements

Some focusable elements may, as a standard, exhibit a particularbehavior in response to directional input. For example, if a pop-up menuhas focus and a user presses a down arrow key, the next element down inthe menu may become highlighted, indicated that it is the currentlyselected menu item. As another example, if a text box has focus and auser presses a right arrow key, a cursor in the text box may shift onecharacter to the right.

In the implementation described, the browser application is implementedto support directional focus navigation while allowing standard behaviorin those focusable elements that may respond in and of themselves todirectional input. To allow this behavior, the browser may beimplemented such that when an element has focus, a user can send a“select” input to indicate that directional inputs from that point onare to be directed to the focused element. A subsequent “select” inputcan be used to indicate that directional inputs from that point are tobe directed to the document in such a way that they drive focusnavigation.

For example, when a pop-up menu element receives the focus, a user maypress the Enter key (or other designated “select” input device) toselect the pop-up menu as the element that will be affected bysubsequent directional input. Once the user has navigated to the desiredmenu item, the user may press the Enter key again to indicate that aselection has been made and to signify to the browser that additionaldirectional inputs will now drive focus navigation. To cancel aselection, the user may press the Escape key (or other designated“cancel” input device).

Exemplary Implementation

FIG. 9 is a block diagram of an example client device 902, displaydevice 904, and various input devices that interact with the clientdevice. Client device 902 may also be referred to as a set-top box, gameconsole or a satellite receiver.

Client device 902 includes a wireless receiving port 906 (e.g., aninfrared (IR) wireless port) for receiving wireless communications froma remote control device 908, a handheld device 910 (such as a personaldigital assistant (PDA) or handheld computer), or other wireless device,such as a wireless keyboard. Additionally, a wired keyboard 912 iscoupled to client device 902 for communicating with the client device.In alternate embodiments, remote control device 908, handheld device910, and/or keyboard 912 may us an RF communication link (or other modeof transmission) to communicate with client 1S device 902. In anotheralternate embodiment, client device 902, display device 904, and aninput device may be implemented as a single hardware unit, such as acellular telephone or as a handheld device similar to handheld device910.

In the exemplary system described herein, directional inputs areprovided by pressing keys arrow keys 914 of remote control 908, arrowkeys or other inputs associated with handheld device 910, or arrow keys916 of keyboard 912. Such keys might comprise four orthogonal directionkeys (up, down, left, and right), or might allow movement in additionaldirections. Alternatively, a particular input device might utilize ajoystick or similar mechanism for directional input, and such an inputdevice might provide increased angular granularity as opposed to thefour-direction granularity provided by four orthogonal direction keys.

Client device 902 includes a rendering engine for generating a videosignal and/or an audio signal, both of which are communicated to displaydevice 904. The video signals and audio signals can be communicated fromclient device 902 to display device 904 via an RF (radio frequency)link, S-video link, composite video link, component video link, or othercommunication link. Although not shown in FIG. 9, a particular clientdevice 902 may include one or more lights or other indicatorsidentifying the current status of the client device. Additionally, aparticular client device 902 may include one or more control buttons orswitches (not shown) for controlling operation of the client device.

FIG. 10 is a block diagram illustrating select components of a clientdevice implemented to support directional focus navigation as describedherein. Client device 902 may be implemented as, but is not limited to,a personal computer system, a cellular telephone, a personal multi-modaldevice, a television, or a television set-top box.

Client device 902 includes a processor 1002, a memory 1004, a displayinterface 1006, an audio output 1008, and a video output 1010. Operatingsystem 1012, rendering engine 1014, and other applications 1016 arestored in memory 1004 and executed on processor 1002. Display interface1006 provides a mechanism for client device 902 to display documentsrendered by rendering engine 1014.

To support directional focus navigation, rendering engine 1014 includesdirectional input interface 1018 for receiving directional input,candidate element identifier 1020 for identifying focusable elementsthat may receive focus based on received directional input, and optimalelement selector 1022 for selecting an optimal focusable element toreceive focus when multiple candidate elements are selected based onreceived directional input.

Conclusion

Although the systems and methods have been described in languagespecific to structural features and/or methodological steps, it is to beunderstood that the invention defined in the appended claims is notnecessarily limited to the specific features or steps described. Rather,the specific features and steps are disclosed as preferred forms ofimplementing the claimed invention.

1. A method implemented on a computer by a processor configured to execute instructions, that, when executed by the processor, direct the computer to perform acts comprising: identifying a first element in a rendered document, the first element having a focus; when a first indication of a directional input corresponding to a first direction is received, identifying a second focusable element that is closest to the first element along the first direction; moving the focus from the first element to the second focusable element in response to the identifying of the second focusable element; and when a second indication of the directional input corresponding to the first direction is received after the first indication and while the second focusable element has focus: identifying a third focusable element and a fourth focusable element closest to the second focusable element along the first direction, the third focusable element being closer to the first element than the fourth focusable element along a second direction that is generally perpendicular to the first direction; and moving the focus from the second focusable element to the third focusable element based in part on the third focusable element being closer to the first element than the fourth focusable element along the second direction, wherein identifying whether an element is closer to another element along a direction comprises comparing either a vertical or horizontal location of the element to a corresponding vertical or horizontal location of the another element.
 2. A method as described in claim 1, wherein the first element, the second focusable element, the third focusable element, and the fourth focusable element are structurally equivalent within the rendered document.
 3. A method as described in claim 1, wherein a plurality of focusable elements are arranged in rows and columns.
 4. A method as described in claim 1, wherein the first direction spans two rendered documents.
 5. A method as described in claim 1, wherein the rendered document is a scrollable document, and the identifying the second focusable element further comprises scrolling the rendered document until the second focusable element is within a displayable area.
 6. A method as described in claim 1, wherein the first direction is vertical and the second direction is horizontal.
 7. A method as described in claim 1, wherein the first direction is horizontal and the second direction is vertical.
 8. A method as described in claim 1, further comprising when a third indication of the directional input corresponding to the first direction is received after the second indication and while the third focusable element has the focus, identifying a fifth focusable element that is closest to the third focusable element along the first direction and closest to the first element along the second direction.
 9. A computer storage media having computer-readable instructions thereon which, when executed by a computer, perform acts comprising: identifying a first element in a rendered document, the first element having a focus; when a first indication of a directional input corresponding to a first direction is received, identifying a second focusable element that is closest to the first element along the first direction; moving the focus from the first element to the second focusable element in response to the identifying of the second focusable element; and when a second indication of the directional input corresponding to the first direction is received after the first indication and while the second focusable element has focus: identifying a third focusable element and a fourth focusable element closest to the second focusable element along the first direction, the third focusable element being closer to the first element than the fourth focusable element along a second direction that is generally perpendicular to the first direction; and moving the focus from the second focusable element to the third focusable element based in part on the third focusable element being closer to the first element than the fourth focusable element along the second direction, wherein: identifying whether an element is closer to another element along a direction comprises comparing either a vertical or horizontal location of the element to a corresponding vertical or horizontal location of the another element.
 10. The computer storage media as described in claim 9, wherein the first element, the second focusable element, the third focusable element, and the fourth focusable element are structurally equivalent within the rendered document.
 11. The computer storage media as described in claim 9, wherein a plurality of focusable elements are arranged in rows and columns.
 12. The computer storage media as described in claim 9, wherein the first direction spans two rendered documents.
 13. The computer storage media as described in claim 9, wherein the rendered document is a scrollable document, and the identifying the second focusable element further comprises scrolling the rendered document until the second focusable element is within a displayable area.
 14. The computer storage media as described in claim 9, wherein the first direction is vertical and the second direction is horizontal.
 15. The computer storage media as described in claim 9, wherein the first direction is horizontal and the second direction is vertical.
 16. The computer storage media as described in claim 9 further comprising when a third indication of the directional input corresponding to the first direction is received after the second indication and while the third focusable element has the focus, identifying a fifth focusable element that is closest to the third focusable element along the first direction and closest to the first element along the second direction. 